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REMARKS 

This is in response to the Office Action mailed on August 24, 2007. Claims 1-21 
were pending, and the Examiner rejected all claims. With this amendment, claims 1, 8, 15 and 
17 are amended. Claims 4-7 are canceled, and the remaining claims are unchanged in the 
application. 

On pages 2-8 of the Office Action, the Examiner rejected all claims 1-21 under 35 
U.S.C. § 102(b) as being anticipated by Huang et al., WO 0193249 (hereinafter Huang). 
Applicant respectfully traverses the Examiner's rejection. 

Claim 1 is directed to a speech processing system that uses a composite language 
model that includes a rules based model portion (such as a context free grammar-CFG) and a 
statistical model portion (such as statistical n-grams) as well as an acoustic model. A decoder is 
coupled to the acoustic model and the composite language model and maps inputs in a natural 
language speech input to pre-terminals and slots of a schema. Therefore, the system not only 
performs speech recognition, but also performs natural language understanding, using the 
composite language model. 

Claim 1 also provides a significant advantage over prior systems in the structure 
of the composite language model. Claim 1 indicates that the statistical model portion has a 
plurality of statistical n-gram models trained based on training data, wherein one statistical n- 
gram model corresponds to each of a plurality of pre-terminals, and also has "a back-off model n- 
gram, separate from the plurality of statistical n-gram models corresponding to the pre-terminals, 
which, when accessed, is configured to assign a backoff score to a word in the vocabulary 
wherein each statistical n-gram model includes a reference to the backoff model portion for all 
unseen words." 

One example of this is graphically shown in FIGS. 18A-18C of the present 
specification. It can easily be seen that only a single backoff probability n-gram is required for 
the entire statistical model portion of the composite model. Each n-gram that corresponds to one 
of the pre-terminals has a reference to the backoff n-gram. Therefore, there is only a single 
backoff n-gram for all unseen words, instead of one for each unseen word in every n-gram in the 
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model. In other words, instead of the n-grams in FIGS. 18A and 18C each having additional arcs 
for a backoff probability distribution, they simply refer to the backoff n-gram shown in FIG. 18B. 

This is a highly advantageous model structure in that it reduces the complexity 
and size of the overall model, hi order to meet these limitations (originally found in claims 6-7), 
the Examiner cited page 15, lines 5-15 of Huang. The relevant portion of this citation simply 
states "an n-gram algorithm may use, for instance, known statistical techniques such as Katz's 
technique, or the binomial posterior distribution backoff technique. In using these techniques, 
the algorithm estimates the probability that a word w(n) will follow a sequence of words wl, w2, 
...w(n-l). These probability values collectively form the n-gram language model." See page 15 
of Huang, lines 8-15. It appears to Applicant that this is the only reference to a backoff 
probability in the entire Huang reference. This simply does not state how the backoff probability 
is structured. There is no mention of a separate n-gram for the backoff probability, much less a 
reference to a separate n-gram from the n-grams that correspond to the per-terminals. Thus, 
Applicant submits that Huang does not anticipate independent claim 1. 

Independent claim 10 is a method of assigning probabilities to word hypotheses 
during speech processing. Claim 10 includes the steps of "assigning an n-gram probability. . . to 
the word hypothesis if the word hypothesis corresponds to a word seen during training of the n- 
gram models; and referring to a separate backoff model for the word hypothesis if the word 
hypothesis corresponds to a word unseen during training of the n-gram model; and assigning a 
backoff probability to each word hypothesis that corresponds to an unseen word, with the backoff 
model." 

In order to meet these limitations, the Examiner cited Huang at page 18, lines 6- 
24. However, this portion of Huang does not teach or suggest or even mention how unseen data 
is treated. It certainly does not mention that it is treated using a separate backoff model and 
assigning a backoff probability to each word hypothesis that corresponds to an unseen word, with 
the backoff model. The cited portion simply neither teaches nor suggests how unseen data is 
treated. Therefore, Applicant submits that independent claim 10 is allowable over Huang. 
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Independent claim 15 is drawn to a composite language model for use is a speech 
recognition system. The composite language model includes "an automatically learned rules- 
based model portion accessed to recognize words in the input speech signal and to map portions 
of an input signal to slots derived from a schema; and a statistical model portion accessed to 
mapped portions of the input speech signal to pre-terminals derived from the schema/' 

It does not appear that Huang teaches that an automatically learned rules-based 
model is used as a language model to perform speech recognition. 

In fact, it appears that Huang mentions where the context free grammar comes 
from on page 22, lines 14-17 where Huang states "...each of the slots can form semantic or 
syntactic concepts in which a context-free grammar is written or otherwise provided.", and at 
page 17, lines 15-23 which state "the manner in which the unified language model is created is 
not essential to the present invention. However, co-pending application entitled CREATING A 
LANGUAGE MODEL FOR A LANGUAGE PROCESSING SYSTEM , filed on June 1, 2000 
and assigned Serial No. 09/585,298 describes various techniques for creating a unified language 
model and is incorporated, . A copy of that reference is provided herewith for the Examiner's 
review. Applicant submits that providing an automatically learned rules-based model as a 
language model to "recognize words in [a] speech signal and to map portions of an input speech 
signal to slots derived from a schema../' is neither taught nor suggested by the references. 
Therefore, Applicant submits that independent claim 15 is allowable. 

Applicant also submits that a number of the dependent claims are independently 
allowable as well. For instance, dependent claim 17 specifically states that the rules-based model 
portion is "an automatically learned context-free grammar (CFG), learned from an example base 
of training data examples/' Again, this simply does not appear to be taught or suggested by the 
references. Therefore, Applicant submits that dependent claim 17 is independently allowable. 

In conclusion, Applicant submits that independent claims 1, 10, and 15 are 
allowable over the reference cited by the Examiner, Applicant further submits that dependent 
claims 2-3, 8-9, 1144 and 16-21, which depend either directly or ultimately from the 
independent claims are allowable as well. Applicant further submits that a number of the 
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dependent claims are independently allowable. Reconsideration and allowance of claims 1-3 and 
8-21 are respectfully requested. 

The Director is authorized to charge any fee deficiency required by this paper or 
credit any overpayment to Deposit Account No. 23-1 123. 

Respectfully submitted, 

WESTMAN, CHAMPLIN & KELLY, PA. 



Bv: /Joseph R. Kelly/ 

Joseph R. Kelly, Reg. No. 34,847 
900 Second Avenue South, Suite 1400 
Minneapolis, Minnesota 55402-3319 
Phone: (612) 334-3222 Fax: (612) 334-3312 
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